Skip to content

Improve plate assay import performance#7124

Merged
labkey-alan merged 2 commits intodevelopfrom
fb_improve_plate_assay_import_perf
Oct 16, 2025
Merged

Improve plate assay import performance#7124
labkey-alan merged 2 commits intodevelopfrom
fb_improve_plate_assay_import_perf

Conversation

@labkey-alan
Copy link
Contributor

Rationale

This PR slightly optimizes the way we query for well samples, and preloads all exp materials for a plate into the materialCache during import, which improves performance.

Benchmarks from my machine show that on develop import time is approximately 58% slower than 25.10. With the changes in this PR import time is approximately 24% slower.

Assay import time is expected to be slower by some amount, because when marking well samples as Material Inputs we are inserting at least 3x the amount of rows into the database, and we are also making additional queries to the Well table during import (in order to determine what samples to mark as inputs).

Related Pull Requests

Changes

  • AssayPlateMetadataServiceImpl.getWellLocationToSampleIdMap: select only the columns we need when querying the well table
  • AbstractAssayTsvDataHandler.checkData: preload all exp materials for a plate into the materialCache

@labkey-alan labkey-alan force-pushed the fb_improve_plate_assay_import_perf branch from 4cb772f to b3867b4 Compare October 15, 2025 22:38
@labkey-alan labkey-alan merged commit 07e812c into develop Oct 16, 2025
11 checks passed
@labkey-alan labkey-alan deleted the fb_improve_plate_assay_import_perf branch October 16, 2025 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants